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We used a brief training procedure that incorporated feedback and role-play practice to train 
staff members to conduct stimulus preference assessments, and we used group-comparison 
methods to evaluate the effects of training. Staff members were trained to implement the 
multiple-stimulus-without-replacement assessment in a single session and the paired-stimulus 
method in another single session. In all 16 cases (2 assessments for 8 trainees), correct responding 
increased to over 80% accuracy; in 14 of those 16 cases, it increased to over 90% accuracy. Thus, 
training produced mastery-level performance in a single training session in almost all cases. 
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Stimulus preference assessments are often a 
necessary first step when developing behavioral 
programs for increasing appropriate behavior. 
Several studies have shown that items identified 
based on the results of preference assessments 
can serve as effective reinforcers for individuals 
with developmental disabilities (DeLeon & 
Iwata, 1996; Fisher et al., 1992; Pace, Ivancic, 
Edwards, Iwata, & Page, 1985). For this reason, 
the ability to conduct a preference assessment is 
an important skill for training staff who work 
with this population. However, only a few 
studies have evaluated training procedures for 
increasing skills involved in conducting prefer- 
ence assessments. Lavie and Sturmey (2002) 
successfully used instructions, modeling, and 
feedback to train staff members to conduct the 
paired-stimulus (PS) preference assessment 
(Fisher et al.) with children. A noteworthy 
feature of this study was that the training 
procedure took only 80 min to conduct. 
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Roscoe, Fisher, Glover, and Volkert (2006) 
later compared the relative effects of feedback 
and contingent money for training staff to 
implement the PS and multiple-stimulus-with- 
out-replacement (MSWO; DeLeon & Iwata, 
1996) assessments. Across training conditions, 
sessions with simulated clients (adult playing 
the role of a client) were alternated with probe 
sessions with real clients (clients who had been 
referred to an outpatient clinic for the treatment 
of problem behavior). Results showed that 
feedback was necessary for facilitating skill 
acquisition, whereas contingent money had 
little effect. In addition, correct responding 
during probe sessions was similar to the 
simulated conditions, demonstrating that train- 
ing using simulated clients readily transferred to 
situations involving real clients. Although the 
study by Roscoe et al. was informative in 
delineating the most useful training component, 
multiple opportunities to practice conducting 
preference assessments were provided prior to 
initiation of the feedback component, making it 
unclear whether rapid increases in performance 
would have been observed if trainees received 
only one or two training sessions prior to the 
feedback component. Therefore, the purpose of 
the current study was to extend this research by 
evaluating the most effective training compo- 
nent identified in Roscoe et al. after only one or 
two baseline sessions. Rapid acquisition after 
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few practice opportunities would demonstrate 
that this training procedure could be imple- 
mented in an efficient manner in routine 
clinical settings with new staff members. 

METHOD 

Participants and Setting 

Trainees. Eight recently hired behavioral 
technicians participated. Each held a bachelor’s 
degree in psychology or a related discipline and 
had some experience working with individuals 
with disabilities. Conducting preference assess- 
ments was a regular component of their initial 
on-the-job training. None had received formal 
training on preference assessments. All had 
provided written informed consent to partici- 
pate and were informed that their performance 
in this study would not be used as an evaluation 
of their job performance. 

Simulated clients, setting, and materials. Sim- 
ulated clients were individuals who played the 
role of clients whose preferences were being 
assessed. All simulated clients had worked in an 
outpatient behavior program for at least 
1 month. Sessions were conducted in a quiet 
room at a day-treatment program for individ- 
uals with developmental disabilities. Materials 
included a video camera and items necessary for 
conducting a preference assessment (i.e., table, 
chairs, leisure items, paper, pen, stopwatch, 
calculator). 

Response Measurement and 
Interobserver Agreement 

Staff members were trained to conduct both 
the PS and MSWO methods. For each trial in 
either the PS or MSWO assessments, there were 
two correct antecedent responses and one to 
three correct consequent responses, described 
below for each assessment method. 

Response definitions fior PS assessment During 
each trial, the observer scored the first anteced- 
ent response as correct if the trainee placed two 
items spaced 30.5 cm apart in front of the client 
and instructed the client to “pick one.” The 


observer scored the consequence as correct if the 
client selected an item within 5 s, and the 
trainee removed the unselected item and 
provided access to the selected item for 5 s 
before initiating a new trial. If the client selected 
both items simultaneously or sequentially, the 
observer scored the consequence as correct if the 
trainee removed both items and reinitiated the 
same trial. If the client did not select an item 
within 5 s, the observer scored a correct 
consequence if the trainee removed both items, 
physically prompted the client to sample the 
items, and reinitiated the same trial. If the client 
did not select an item within 5 s or selected 
both items a second time, the observer scored a 
correct consequence if the trainee removed both 
items and initiated the next trial. If the client 
grabbed an item that was not presented, the 
observer scored a correct consequence if the 
trainee blocked access to or removed the item 
and continued with the current trial. Finally, at 
the end of the session, the trainee’s data sheet 
was examined and one correct response was 
scored if the trainee recorded client selections 
on each trial, and a second correct response was 
scored if the trainee accurately summarized the 
client data, which consisted of obtaining a 
selection percentage and corresponding rank for 
each item. 

Response defiinitions fior MSWO assessment. 
Before the first trial, the observer scored a 
correct antecedent response if the trainee singly 
presented each item to the client for 30 s. Next, 
the observer scored a correct antecedent re- 
sponse if the trainee placed all items in a straight 
line or arc in front of the client and instructed 
the client to “pick one.” If the client selected an 
item within 30 s, the observer scored a correct 
consequence if the trainee provided access to 
that item for 30 s. After 30 s, the observer 
scored a correct consequence if the trainee 
removed the selected item, did not put the item 
back into the array, and rotated the remaining 
items. If the client selected two items sequen- 
tially, the observer scored a correct consequence 
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if the trainee gave the client access to the item 
selected first for 30 s. If the client selected two 
items simultaneously, the observer scored a 
correct consequence if the trainee blocked access 
to both items and reinitiated the same trial. If 
the client again simultaneously selected two 
items, the observer scored a correct consequence 
if the trainee removed all items and initiated a 
new session. If the client did not select an item 
within 30 s, the observer scored a correct 
consequence if the trainee removed all items 
and initiated a new session. If the client grabbed 
another item while he or she had access to the 
one selected, the observer scored a correct 
consequence if the trainee blocked access to 
the item and continued with the current trial. 
Finally, at the end of the session, the trainee’s 
data sheet was examined and one correct 
response was scored if the trainee recorded 
client selections on each trial, and a second 
correct response was scored if the trainee 
accurately summarized the client data, which 
consisted of obtaining a selection percentage 
and corresponding rank for each item. 

Scripts used during simulated assessments. The 
simulated clients followed one of three scripts 
when the trainees conducted the preference 
assessments. The three scripts for each type of 
preference assessment specified client responses 
for 1 6 preference assessment trials: five standard 
trials and 1 1 distracter trials. Standard trials 
were ones in which the simulated client selected 
one item within the first 5 s of the first or 
second presentation of the pair (PS assessment) 
or within 30 s of the presentation of the array of 
items (MSWO). During distracter trials, simu- 
lated client responses included (a) simulta- 
neously selecting two stimuli (four trials), (b) 
selecting two stimuli sequentially in quick 
succession (one trial), (c) grabbing a stimulus 
that was not in the array (one trial), and (d) not 
selecting a stimulus within the appropriate time 
period (five trials). We exposed all trainees to all 
three scripts, which were ordered randomly 
across sessions. 


Interobserver agreement. Data were collected 
in vivo by trained observers who scored target 
responses with a paper and pencil; however, we 
videotaped all sessions to obtain interobserver 
agreement data. The dependent variable was the 
number of responses performed correctly (as 
defined above) divided by all possible responses 
for the first 16 trials, multiplied by 100%. A 
second observer independently collected data on 
50% of the videotaped sessions. Observers’ 
records were compared for each trial in which a 
response was recorded by one of the observers. 
An agreement was defined as both observers 
scoring the same response (either correct or 
incorrect). The number of agreements was 
divided by the number of agreements plus 
disagreements and multiplied by 100% to 
obtain a percentage agreement score. The mean 
agreement score for trainee behavior was 94% 
(range, 80% to 100%). 

Experimental Design 

We asked trainees to conduct a PS assessment 
in one condition and a MSWO assessment in 
the other condition across phases in accordance 
with a multielement design. During the baseline 
phase, the same procedure was conducted with 
all ttainees. During the subsequent two phases, 
4 trainees were assigned randomly to Group 1 
and 4 were assigned randomly to Group 2. 
During the second phase. Group 1 received 
training for the MSWO assessment prior to the 
MSWO condition but were given no instruc- 
tions or training for the PS assessment prior to 
the PS condition. Group 2 received training for 
the PS assessment prior to the PS condition but 
were given no instructions or training for the 
MSWO assessment prior to the MSWO 
condition. During the third phase, both groups 
received training for both assessments prior to 
both assessment conditions. 

Training 

Baseline (written instruction). During all 
conditions, we instructed trainees to formulate 
a list of the client’s most to least preferred items. 
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We gave trainees brief summaries of the PS and 
MSWO assessments from the method sections 
of Fisher et al. (1992) and DeLeon and Iwata 
(1996) for 30 min prior to sessions. Data sheets 
for each assessment also were provided. 

Training (feedback and role-play practice). 
Immediately prior to each session, the experi- 
menter reviewed the videotape and data sheet 
from the preceding session with the trainee (we 
used the baseline session for the first feedback 
session). The experimenter provided feedback 
by noting whether or not each target behavior 
depicted on the video and data sheet was 
performed correctly. Role playing was included 
to ensure multiple exposures to feedback within 
one training session. During role playing, the 
experimenter (playing the client) demonstrated 
each potential client response that may occur as 
indicated on the scripts. The trainee was 
instructed to respond (the correct consequent 
event was not specified). Immediately following 
trainee performances of each consequent event, 
the experimenter presented feedback in the 
manner described above. Training sessions were 
15 to 20 min long. 

RESULTS AND DISCUSSION 

Figure 1 shows the results for Group 1 (who 
received the MSWO training first) and for 
Group 2 (who received the PS training first). 
During the written instruction phase, trainees 
showed moderate levels of correct performance 
when implementing the MSWO assessment in 
both Group 1 {M = 44%; range, 29% to 67%) 
and Group 2 {M = 47%; range, 43% to 52%). 
On the PS assessment, written instruction was 
associated with slightly better performance for 


Group 1 {M = 47%; range, 31% to 70%) than 
for Group 2 {M = 34%; range, 28% to 45%). 

Following training on the MSWO assess- 
ment, the mean performance of Group 1 
increased to 96% (range, 93% to 99%), whereas 
without training, Group 2’s performance in- 
creased to a mean of only 54% (range, 43% to 
62%). The difference in performance between 
the trained (Group 1) and untrained (Group 2) 
trainees on the second administration of the 
MSWO assessment was statistically significant 
{F = 70.6; p < .001; df= 1,5; with baseline 
scores as covariates). Following training on the 
PS assessment, the mean performance of Group 
2 increased to 94% (range, 92% to 96%), 
whereas without training, Group 1 showed a 
slight decrease in performance to a mean of 
46% (range, 38% to 50%). The difference in 
performance between the trained (Group 2) and 
untrained (Group 1) trainees on the second 
administration of the PS assessment was 
statistically significant {F = 22G.7-, p < .0001; 
df = 1, 5; with baseline scores as covariates). 
After all trainees received training on both 
assessments, the mean performance across all 
trainees was 95% (range, 81% to 100%) for the 
MSWO assessment and 96% (range, 85% to 
100%) for the PS assessment. Inspection of 
individual data shows that each trainee showed 
small or no changes in their performances on 
the MSWO and PS assessments without 
training (mean change of 2 percentage points; 
range, —21 to 17 percentage points) and large 
improvements in their performances following 
training (mean change of 56 percentage points; 
range, 32 to 66 percentage points). 

Overall, these findings indicated that an 
efficient and effective package could be devel- 


Figure 1 . The effects of feedback and role-play training during a single training session on the percentage of correct 
responses when implementing the PS and MSWO preference assessments. Group 1 (left) was trained on MSWO 
between the first and second phases and on PS between the second and third phases. Group 2 (right) was trained on PS 
between the first and second phases and on MSWO between the second and third phases. 
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oped to train new staff members in a single 
session to conduct the MSWO or PS assess- 
ments accurately. The current study replicates 
and extends the findings of Roscoe et al. (2006) 
by demonstrating that the most effective 
component identified (i.e., feedback) could be 
incorporated into an efficient training proce- 
dure that combined feedback and role-play 
practice for staff orientation and training 
programs in routine clinical settings. Correct 
responding increased to over 80% in all 16 cases 
(two assessments for 8 trainees) and over 90% 
in 14 cases. Thus, the training procedures 
produced mastery-level performance in a single 
session in almost all cases. 

These outcomes could have been strength- 
ened by demonstrating that training using 
simulated clients transferred to situations in- 
volving real clients. Although the probe data 
with real clients in Roscoe et al. (2006) 
demonstrated transfer of training, probes often 
followed multiple training sessions. Future 
research should examine the effects of the 
training procedure used in the present study 
with real clients after a single training session. 
Also, the training procedure might have been 
more efficient if it had been conducted in a 
group format. However, because it may not 


always be feasible to train several staff members 
concurrently (due to varying hire dates), it 
seemed to be more informative to evaluate the 
effects of within-participant training. 
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